Information processing apparatus, information processing system, control method, and program

ABSTRACT

An information processing apparatus (2000) acquires a shelf rack image (12) in which a product shelf rack on which a product is displayed is imaged. The information processing apparatus (2000) performs image analysis on the shelf rack image (12), and generates information (actual display information) relevant to a display situation of the product on a product shelf rack (20). The information processing apparatus (2000) acquires reference display information representing a reference for display of the product on the product shelf rack (20). The information processing apparatus (2000) compares the actual display information generated by performing the image analysis on the shelf rack image (12) with the acquired reference display information, and generates comparison information representing a result.

This application is a Continuation application of Ser. No. 16/651,762,filed Mar. 27, 2020, which is a National Stage of InternationalApplication No. PCT/JP2018/033600, filed Sep. 11, 2018, claimingpriority to Japanese Patent Application No. 2017-191790, filed Sep. 29,2017, the contents of all of which are incorporated herein by referencein their entirety.

TECHNICAL FIELD

The present invention relates to management of a product.

BACKGROUND ART

In stores, such as convenience stores and supermarkets, products aredisplayed on product shelves. An information processing technology hasbeen developed to increase efficiency of an operation of displaying theproducts. For example, Patent Literature 1 discloses a technology forautomatically determining whether or not a relationship between aproduct placed on a product shelf rack and a shelf label attached to theproduct shelf rack is correct.

RELATED DOCUMENT Patent Document

-   [Patent Document 1] International Publication No. WO 2016/052383

SUMMARY OF THE INVENTION Technical Problem

There is a case where an operation is performed of prescribing a methodfor displaying each product on a product shelf rack (a display position,a number of faces, or the like) in advance, and displaying the productaccording to the prescription. However, even though the operation isperformed, there is a case where the product is not actually displayedaccording to the prescription. Here, a work is performed in which aperson or the like who is in charge of a manufacturer visually checks adisplay situation of the product and checks whether or not the productis correctly displayed. However, in a case where the check is visuallyperformed, there is a problem in that a work load of a worker is large.

The present invention has been made in view of the above problems, andone object of the present invention is to provide a technology which iscapable of easily grasping correctness of display of the product.

Solution to Problem

An information processing apparatus according to the present inventionincludes: 1) an image analysis unit that performs image analysis on ashelf rack image in which a product shelf rack is captured, andgenerates actual display information representing a display situation ofa product on the product shelf rack, and 2) a comparison unit thatcompares the actual display information with reference displayinformation representing a reference for display of the product on theproduct shelf rack, and generates comparison information representing aresult of the comparison.

An information processing system according to the present inventionincludes an information processing apparatus and a mobile terminal.

The mobile terminal transmits a shelf rack image, in which a productshelf rack is captured, to the information processing apparatus.

The information processing apparatus includes: 1) an image analysis unitthat receives the transmitted shelf rack image, performs image analysison the shelf rack image, and generates actual display informationrepresenting a display situation of the product on the product shelfrack, 2) a comparison unit that compares the actual display informationwith reference display information representing a reference for displayof the product on the product shelf rack, and generates comparisoninformation representing a result of the comparison, and 3) a generationunit that generates a result image representing a product which does notmatch the reference indicated in the reference display information and aproduct which matches the reference indicated in the reference displayinformation in a distinguishable aspect using the result of thecomparison, and transmits the generated result image to a mobileterminal.

The mobile terminal receives the result image and displays the resultimage on a display apparatus.

A control method of the present invention is executed by a computer. Thecontrol method includes: 1) performing image analysis on a shelf rackimage in which a product shelf rack is captured, and generating actualdisplay information representing a display situation of a product on theproduct shelf rack; and 2) comparing the actual display information withreference display information representing a reference for display ofthe product on the product shelf rack, and generating comparisoninformation representing a result of the comparison.

A program of the present invention causes a computer to executerespective steps included in the control method of the presentinvention.

Advantageous Effects of Invention

According to the present invention, there is provided a technology whichis capable of easily grasping correctness of display of a product.

BRIEF DESCRIPTION OF THE DRAWINGS

The above-described object and other objects, features, and advantageswill be further clarified by preferred example embodiments describedbelow and the accompanying drawings attached thereto.

FIG. 1 is a diagram conceptually illustrating an operation of aninformation processing apparatus according to a first exampleembodiment.

FIG. 2 is a diagram illustrating a functional configuration of theinformation processing apparatus.

FIG. 3 is a diagram illustrating a configuration of a computer thatrealizes the information processing apparatus.

FIG. 4 is a diagram illustrating a usage pattern of the informationprocessing apparatus.

FIG. 5 is a flowchart illustrating a flow of a process executed by theinformation processing apparatus according to the first exampleembodiment.

FIG. 6 is a diagram illustrating a case where the same product isseparately displayed.

FIG. 7 is a diagram illustrating a configuration of actual displayinformation.

FIG. 8 is a diagram illustrating a configuration of reference displayinformation.

FIG. 9 is a diagram illustrating comparison information.

FIG. 10 is a block diagram illustrating a functional configuration of aninformation processing apparatus according to a second exampleembodiment.

FIG. 11 is a first diagram illustrating a result image.

FIG. 12 is a second diagram illustrating the result image.

FIG. 13 is a third diagram illustrating the result image.

FIG. 14 is a diagram illustrating an example of the result imageincluding a match rate for each shelf rack.

FIG. 15 is a diagram illustrating an example of the result imageincluding a list of products.

FIG. 16 is a diagram illustrating a state in which the result imagechanges according to an operation with respect to the list of theproducts.

FIG. 17 is a diagram illustrating a state in which the result imagechanges according to an operation with respect to an image of theproduct.

FIG. 18 is a diagram illustrating an example of the result imageincluding superimposition data representing an unknown product.

FIG. 19 is a diagram illustrating correction information.

DESCRIPTION OF EMBODIMENTS

Hereinafter, example embodiments of the present invention will bedescribed with reference to the accompanying drawings. The samereference numerals are given to the same components throughout thedrawings, and the description thereof will not be repeated. Further,unless being particularly specified, in each block diagram, each blockrepresents not a configuration in a hardware unit but a configuration ina functional unit.

First Example Embodiment Overview

FIG. 1 is a diagram conceptually illustrating an operation of aninformation processing apparatus (an information processing apparatus2000 whose configuration is illustrated in FIG. 2 ) according to a firstexample embodiment. Here, the operation of the information processingapparatus 2000 described with reference to FIG. 1 is an example foreasily understanding the information processing apparatus 2000, and doesnot limit the operation of the information processing apparatus 2000.Details and variations of the operation of the information processingapparatus 2000 will be described later.

The information processing apparatus 2000 acquires a shelf rack image inwhich a product shelf rack, on which products are displayed, iscaptured. In FIG. 1 , a shelf rack image 12 is generated as a resultthat a product shelf rack 20 is imaged by a camera 10. The shelf rackimage 12 includes the product shelf rack 20 and the products displayedon the product shelf rack 20. It should be noted that, the product shelfrack, which is a target of the information processing apparatus 2000,may be a place where the products are displayed, and is not limited to aproduct shelf rack which includes a plurality of stages as illustratedin FIG. 1 . For example, in a case where products are stacked flat on adesk, this desk is treated as a product shelf rack.

The information processing apparatus 2000 acquires reference displayinformation representing a reference for display of the product on theproduct shelf rack 20. For example, the reference display informationindicates information relevant to a product to be displayed on theproduct shelf rack 20, a position of each product on the product shelfrack 20, and the like.

The information processing apparatus 2000 compares an actual displaysituation represented by the shelf rack image 12 with the referencedisplay information, and generates information representing a result ofthe comparison. For example, in FIG. 1 , information, which indicatesthat an error occurs in disposition of two products, is generated.

In order to realize the above-described processing, the informationprocessing apparatus 2000 performs image analysis on the shelf rackimage 12, and generates information (hereinafter, actual displayinformation) relevant to a display situation of the products on theproduct shelf rack 20. The actual display information indicates, forexample, names of the products displayed on the product shelf rack 20and positions of the respective products.

The information processing apparatus 2000 compares the actual displayinformation generated by performing image analysis on the shelf rackimage 12 with the acquired reference display information, and generatescomparison information representing the result.

Effects

According to the information processing apparatus 2000 of the presentexample embodiment, the actual display information generated byperforming the image analysis on the shelf rack image 12 is comparedwith the reference display information, and thus the comparisoninformation representing a result of the comparison is generated. Theactual display information is compared with the reference displayinformation as above, and thus it is possible to grasp whether or not adisplay situation of the products on the product shelf rack 20 matchesthe reference. Therefore, it is possible to easily grasp whether or notthe display situation of the products matches the reference, compared toa case where a person visually checks whether or not the displaysituation of the products matches the reference.

In addition, in the case where the person visually checks the displaysituation, there is a problem in that a human error, such as a mistakein vision, occurs. According to the information processing apparatus2000, it is possible to prevent occurrence of the human error.

Example of Functional Configuration

FIG. 2 is a diagram illustrating a functional configuration of theinformation processing apparatus 2000. The information processingapparatus 2000 includes an image analysis unit 2020 and a comparisonunit 2040. The image analysis unit 2020 generates the actual displayinformation by performing the image analysis on the shelf rack image 12.The comparison unit 2040 generates the comparison information bycomparing the actual display information with the reference displayinformation.

Example of Hardware Configuration of Information Processing Apparatus2000

Each functional component of the information processing apparatus 2000may be realized by hardware (for example: a hard-wired electroniccircuit) that implements each functional component, or a combination ofthe hardware and software (for example: a combination of the electroniccircuit and a program which controls the electronic circuit).Hereinafter, a case where each functional component of the informationprocessing apparatus 2000 is realized by the combination of the hardwareand the software will be further described.

A computer 1000 is a variety of computers. For example, the computer1000 is a personal computer (PC), a server machine, a tablet terminal, asmartphone, or the like. The computer 1000 may be a dedicated computerdesigned for realizing the information processing apparatus 2000 or ageneral-purpose computer.

FIG. 3 is a diagram illustrating a configuration of the computer 1000that realizes the information processing apparatus 2000. The computer1000 includes a bus 1020, a processor 1040, a memory 1060, a storagedevice 1080, an input and output interface 1100, and a network interface1120. The bus 1020 is a data transmission path through which theprocessor 1040, the memory 1060, the storage device 1080, the input andoutput interface 1100, and the network interface 1120 transmit andreceive data to and from each other. However, a method for connectingthe processor 1040 and the like to each other is not limited to busconnection. The processor 1040 is a processor configured with a CentralProcessing Unit (CPU), a Graphics Processing Unit (GPU), or the like.The memory 1060 is a main storage apparatus which is configured with aRandom Access Memory (RAM) or the like. The storage device 1080 is anauxiliary storage apparatus configured with a hard disk, a Solid StateDrive (SSD), a memory card, a Read Only Memory (ROM), or the like. Theinput and output interface 1100 is an interface for connecting thecomputer 1000 to an input and output device. For example, an inputdevice, such as a keyboard, and an output device, such as a displayapparatus, are connected to the input and output interface 1100. Thenetwork interface 1120 is an interface for connecting the computer 1000to a network. The communication network is, for example, a Local AreaNetwork (LAN) or a Wide Area Network (WAN). A method for connecting tothe network by the network interface 1120 may be wireless connection orwired connection.

The storage device 1080 stores program modules that implement respectivefunctions of the information processing apparatus 2000. The processor1040 realizes the respective functions corresponding to the programmodules by executing the program modules.

<Camera 10>

The camera 10 is an arbitrary imaging device that is capable ofperforming imaging and generating an image representing a result of theimaging. For example, the camera 10 is a camera provided in asmartphone, a tablet terminal, or the like.

Example of Usage Pattern of Information Processing Apparatus 2000>

For example, a user of the information processing apparatus 2000 usesthe information processing apparatus 2000 via a mobile terminal. Theuser of the information processing apparatus 2000 is a person, such as arounder, who performs an operation of checking the display situation ofthe products. FIG. 4 is a diagram illustrating a usage pattern of theinformation processing apparatus 2000. In FIG. 4 , an informationprocessing system 4000 includes the information processing apparatus2000 and a mobile terminal 3000. The mobile terminal 3000 is a mobileterminal used by the user. The information processing apparatus 2000 isrealized, for example, as a server apparatus.

The mobile terminal 3000 is provided with the camera 10. In addition, anapplication 100 for using the information processing apparatus 2000 isinstalled in the mobile terminal 3000 in advance. The application 100controls the camera 10 to perform imaging. Further, the application 100transmits a request including the shelf rack image 12 generated by thecamera 10 to the information processing apparatus 2000. The informationprocessing apparatus 2000 receives the request from the application.

The information processing apparatus 2000 performs the image analysis onthe shelf rack image 12 indicated in the request to generate the actualdisplay information, and compares the actual display information withthe reference display information to generate the comparisoninformation. Then, the information processing apparatus 2000 transmitsthe generated comparison information to the mobile terminal 3000.

It should be noted that, the comparison information is not necessarilytransmitted to the mobile terminal 3000. A method for treating thegenerated comparison information will be described later. Further, theinformation processing apparatus 2000 is not necessarily used via themobile terminal 3000.

<Process Flow>

FIG. 5 is a flowchart illustrating a flow of a process executed by theinformation processing apparatus 2000 according to the first exampleembodiment. The image analysis unit 2020 acquires the shelf rack image12 (S102). The image analysis unit 2020 generates the actual displayinformation by performing the image analysis on the shelf rack image 12(S104). The comparison unit 2040 acquires the reference displayinformation (S106). The comparison unit 2040 generates the comparisoninformation by comparing the actual display information with thereference display information (S108).

It should be noted that, the flow of the process executed by theinformation processing apparatus 2000 is not limited to the aboveexample. For example, the acquisition of the reference displayinformation may be performed before the actual display information isgenerated.

Timing at which the information processing apparatus 2000 executes aseries of processes illustrated in FIG. 5 is arbitrary. For example, theinformation processing apparatus 2000 executes the series of processesillustrated in FIG. 5 in response to reception of the above-describedrequest from the application 100. In addition, for example, theinformation processing apparatus 2000 may receive an input operation ofspecifying the shelf rack image 12 from the user who operates the inputdevice connected to the information processing apparatus 2000. In thiscase, the information processing apparatus 2000 executes the series ofprocesses illustrated in FIG. 5 triggered by reception of an inputoperation from the user.

<Method for Acquiring Shelf Rack Image 12: S102>

A method for acquiring the shelf rack image 12 by the image analysisunit 2020 is arbitrary. For example, the image analysis unit 2020acquires the shelf rack image 12 included in the request transmitted bythe above-described mobile terminal 3000. In addition, for example, inresponse to the reception of the operation of specifying the shelf rackimage 12 by the user, the image analysis unit 2020 may access thestorage apparatus, in which the specified shelf rack image 12 is stored,and may acquire the shelf rack image 12.

Here, one product shelf rack 20 may be represented by a plurality ofshelf rack images 12. For example, in a case where the product shelfrack 20 is long horizontally, it is possible to image whole parts of theproduct shelf rack 20 by imaging the product shelf rack 20 a pluralityof times while shifting an imaging range of the camera 10 in ahorizontal direction.

At this time, in order to ensure that all products displayed on theproduct shelf rack 20 are imaged, for two shelf rack images 12 includingregions adjacent to each other on the product shelf rack 20, it ispreferable to image the product shelf rack 20 such that a product on aboundary (ends of the shelf rack images 12) is included in both theshelf rack images 12. For example, it is possible for the informationprocessing apparatus 2000 to grasp the situation of one product shelfrack 20 from the plurality of shelf rack images 12 by treating theplurality of shelf rack images 12 as being combined at parts includingcommon products. This corresponds to, for example, a case where aplurality of pictures are taken such that two pictures partially overlapeach other to obtain a panoramic picture.

<Method for Identifying Product Shelf Rack 20 Included in Shelf RackImage 12>

The information processing apparatus 2000 needs to identify whichproduct shelf rack installed in a store or the like is the product shelfrack 20 included in the acquired shelf rack image 12. For example, whenthe shelf rack image 12 is acquired, the information processingapparatus 2000 acquires an identifier that identify the product shelfrack 20 included in the shelf rack image 12.

For example, as described above, it is assumed that a request includingthe shelf rack image 12 is transmitted from the application 100, whichoperates on the mobile terminal 3000, to the information processingapparatus 2000. In this case, the application 100 includes an identifierof the product shelf rack 20 included in the shelf rack image 12 in therequest. For example, the application 100 receives an input forspecifying the identifier of the product shelf rack 20 from the user ofthe mobile terminal 3000, and includes the input identifier in therequest.

A method for identifying the product shelf rack 20 included in the shelfrack image 12 is not limited to a method for acquiring the identifier ofthe product shelf rack 20 together with the shelf rack image 12. Forexample, the identifier of the product shelf rack 20 is caused to beimaged by the camera 10. Specifically, a mark representing theidentifier of the product shelf rack 20 is attached to the product shelfrack 20 in advance. In this manner, the mark representing the identifierof the product shelf rack 20 is included in the shelf rack image 12generated as a result of imaging the product shelf rack 20 by the camera10. It is possible for the information processing apparatus 2000 toidentify the product shelf rack 20 included in the shelf rack image 12by performing the image analysis on the mark. For example, it ispossible to use a character string representing the identifier of theproduct shelf rack 20, a QR code (registered trademark), or the like, asthe mark.

<Generation of Actual Display Information: S104>

The image analysis unit 2020 generates the actual display information byperforming the image analysis on the shelf rack image 12 (S104).Hereinafter, information included in the actual display information willbe described.

<<Identification Information of Product>>

The actual display information includes identification information ofeach product displayed on the product shelf rack 20. For example, theidentification information of the product is a product name, anidentifier, such as a Japanese Article Number (JAN) code, or the like.Hereinafter, unless otherwise specified, description will be given onthe assumption that the product name is used as the identificationinformation of the product. The same applies to the reference displayinformation which will be described later.

For example, the image analysis unit 2020 extracts an image regionrepresenting each product from the shelf rack image 12. The imageanalysis unit 2020 acquires information relevant to the productrepresented by the image region from a product database by comparing theextracted image region with the image feature of the product registeredin the product database. In the product database, it is assumed that,for each product, any of pieces of information, such as at least theproduct name, the identifier (such as the Japanese Article Number (JAN)code), a name (for example, a brand name) of a group to which theproduct belongs, a product category to which the product belongs, and acontent of the product (for example, 350 ml), a size of the product, astandard price of the product, and manufacturer information isregistered. In addition, a manufacturer of the product is an arbitrarymanufacturer (for example, a manufacturer or a sales manufacturer)related to the product.

It is possible for the image analysis unit 2020 to identify the productname of each product included in the shelf rack image 12 based on theinformation acquired from the product database as described above.

<<Group Name>>

In addition, for example, the actual display information indicates aname of a group to which the product belongs. For example, similar tothe above-described product name, it is possible to acquire the group,to which the product belongs, from the product database. It is possibleto use any information, such as the identifier of the manufacturer,which is capable of identifying the manufacturer, instead of the name ofthe group. The same applies to the reference display information whichwill be described later.

<<Manufacturer Name>>

In addition, for example, the actual display information indicates amanufacturer of the product. For example, similar to the above-describedproduct name, it is possible to acquire the manufacturer of the productfrom the product database. It is possible to use any information, suchas an identifier of the manufacturer, which is capable of identifyingthe manufacturer, instead of a manufacturer name. The same applies tothe reference display information which will be described later.

<<Position>>

In addition, for example, the actual display information indicates aposition of the product on the product shelf rack 20. The position ofthe product is indicated by, for example, a combination of a position ina stage direction (for example, the product is positioned on what stagefrom a top stage) and a position in a column direction (for example, theproduct is positioned in what column from a leftmost column). Here, itis possible to use an existing technology as a technology fordetermining the position of each product from an image including theproduct shelf rack.

In a case where the same type of product (product having the sameproduct name) is disposed at a plurality of positions, the actualdisplay information indicates the plurality of positions for oneproduct.

<<Number of Faces>>

In addition, for example, the actual display information indicates thenumber of faces of the product. It should be noted that, there may be acase where the same product is separately displayed on the product shelfrack 20. FIG. 6 is a diagram illustrating a case where the same productis separately displayed. In FIG. 6 , product A is separately disposed inan area 70-1 and an area 70-2. The number of faces in the area 70-1 istwo, and the number of faces in the area 70-2 is three.

In the case where the same product is disposed in places separated fromeach other in this way, for example, the actual display informationindividually indicates the number of faces for the respective placesseparated from each other. For example, in the case of FIG. 6 , theactual display information indicates, as the number of faces of theproduct A, 2 for the area 70-1, and indicates 3 for the area 70-2.

In addition, for example, in the actual display information, a valueacquired by summing the faces in the plurality of places separated fromeach other may be used as the number of faces of the products. Forexample, in the case of FIG. 6 , the actual display informationindicates 5 as the number of faces of the product A.

<<Example of Configuration of Actual Display Information>>

FIG. 7 is a diagram illustrating a configuration of the actual displayinformation. The display information in FIG. 7 is expressed as a table400. The table 400 includes columns of a product 402, a group 404, amanufacturer 406, a position 408, and the number of faces 410.

Here, information to be included in the actual display information, thatis, information to be acquired by performing the image analysis on theshelf rack image 12 may be set in the image analysis unit 2020 inadvance, or may be dynamically determined. In the latter case, forexample, the image analysis unit 2020 determines the information to beacquired from the shelf rack image 12 based on the acquired referencedisplay information. For example, in a case where the reference displayinformation indicates a product to be placed on the shelf rack and aposition where each product should be placed, the information to beacquired from the shelf rack image 12 by the image analysis unit 2020 isthe product name and the position of the product.

<Reference Display Information>

As described above, the reference display information indicates thereference for the display of the product on the product shelf rack 20.Hereinafter, information indicated in the reference display informationwill be described.

<<Identification Information of Product>>

For example, the reference display information indicates identificationinformation (including the product name, the JAN code, or the like) ofeach product to be displayed on the product shelf rack 20. In thedescription below, unless otherwise specified, it is assumed that theidentification information of the product indicated by the referencedisplay information is the product name.

0<<Position>>

In addition, for example, the reference display information indicates aposition where each product should be placed on the product shelf rack20. The position of the product indicated by the reference displayinformation is represented by, for example, the combination of theposition in the stage direction and the position in the columndirection. In this case, the reference display information indicatesthat both the position of the product in the stage direction and theposition in the column direction should be the positions indicated bythe reference display information.

The position of the product indicated by the reference displayinformation may be a range of the position in the column direction or avertical direction. For example, the reference display informationindicates a range such as “second to fourth stages” or “third to fifthcolumns”. In addition, for example, the reference display informationmay indicate only a position in the stage direction for a certainproduct. In this case, it is possible to display the product in anycolumn in the stage indicated in the reference display information.Similarly, the reference display information may indicate only theposition in the column direction for a certain product.

<<Number of Faces>>

In addition, for example, the reference display information indicatesthe number of faces of the product. For example, a case where thereference display information indicates 5 as the number of faces of theproduct A indicates that the product A should be displayed such that thenumber of faces is 5.

<<Parallel Request>>

In addition, for example, the reference display information may indicatethat, for a certain product, the product should be collectively disposed(parallel request). In other words, it means that the product, for whichthe parallel request is indicated in the reference display information,should not be separately disposed in a plurality of areas, asillustrated in FIG. 6 .

Here, a unit, in which the parallel request is specified, is not limitedto the product. For example, the parallel request may be specified inunits of a product group. In this case, for example, the referencedisplay information indicates the parallel request for products of adetermined brand. Each product, which belongs to the brand for which theparallel request is indicated in the reference display information, isrequired to be disposed at a collective position on the product shelfrack 20.

In addition, for example, the parallel request may be specified in unitsof a manufacturer. In this case, for example, the reference displayinformation indicates the parallel request for a product of a determinedmanufacturer. Each product of the manufacturer, for which the parallelrequest is indicated in the reference display information, is requiredto be disposed at the collective position on the product shelf rack 20.

In addition, for example, the parallel request may be specified by thecontent of the product. In this case, for example, the reference displayinformation indicates the parallel request for a product having adetermined content. Each product, for which the parallel request isindicated in the reference display information and which has adetermined content, is required to be disposed at the collectiveposition on the product shelf rack 20. More specifically, for example,in a case where the determined content indicates 500 ml, a producthaving a content of 500 ml, for example, a canned beer of 500 ml sold bythe manufacturer is disposed at a collective position on the productshelf rack 20.

<<Arrangement Request>>

In addition, for example, the reference display information may indicatea request (hereinafter, an arrangement request) relevant to adisposition relationship among a plurality of products. For example, thearrangement request is a request to “dispose the product A and a productB to be adjacent to each other”. For example, it is assumed that thereference display information indicates the arrangement request withoutindicating the positions for the product A and the product B. In thiscase, the product A and the product B may be disposed at arbitrarypositions on the product shelf rack 20 as long as they are adjacent toeach other.

In contrast, the arrangement request may indicate that multiple productsshould be separately placed. For example, it is conceivable to define anarrangement request such as “the product A and the product B aredisposed in different stages”.

Here, a unit, in which the arrangement request is specified, is notlimited to the product. For example, the arrangement request may bespecified in units of a product group. For example, there is a requestto “dispose a group D and a group E to be adjacent to each other”. Inthis case, it is necessary that, in the product shelf rack 20, a regionwhere the products belonging to the group D are disposed and a regionwhere the products belonging to the group E are disposed are adjacent toeach other. It should be noted that, similar to the product arrangementrequest, the group arrangement request may indicate that a plurality ofgroups should be separately disposed.

In addition, for example, the arrangement request may be specified inunits of a manufacturer. In this case, for example, there is a requestto “dispose a product of a manufacturer X and a product of amanufacturer Y to be adjacent to each other”. In this case, it isnecessary that, in the product shelf rack 20, a region where the productof the manufacturer X and a region where the product of the manufacturerY are disposed to be adjacent to each other. It should be noted that,similar to the product arrangement request, the manufacturer arrangementrequest may indicate that the products of a plurality of manufacturersshould be disposed to be separated from each other.

<<Applicable Period>>

In addition, for example, the reference display information mayindicate, for each product, a period during which the reference displayinformation is applied to the product. For example, a case where thereference display information indicates an applicable period of “Oct. 1,2017 to Oct. 31, 2017 ” for the product A means that it is necessary toobserve information (for example, the position) for the product Aindicated by the reference display information in the applicable period.The applicable period may be defined for a group of the products or theentire reference display information, instead of each product.

Example of Configuration of Reference Display Information

FIG. 8 is a diagram illustrating a configuration of the referencedisplay information. A table in FIG. 8 is expressed as a table 500. Thetable 500 includes columns of a product 502, a group 504, a manufacturer506, a position 508, the number of faces 510, a parallel request 512, anarrangement request 514, and an applicable period 516.

Data is indicated in any one of the product 502, the group 504, and themanufacturer 506. A record, in which the product name is indicated inthe product 502, indicates a reference relevant to the product havingthe name. A record, in which the group name is indicated in the group504, indicates a reference relevant to the group having the name. Arecord, in which the manufacturer name is indicated in the manufacturer506, indicates a reference relevant to the manufacturer having the name.A product to be disposed on the product shelf rack 20 is a product whoseproduct name is indicated in the product 502. It should be noted that,the table 500 described with reference to FIG. 8 is not limited to theabove-described configuration. For example, the table 500 may furtherinclude at least one of the identifier, the product category, thecontent, and the size of the product.

The reference display information may be information relevant to allproducts placed on the product shelf rack 20 or information relevant tosome products. In the latter case, for example, the reference displayinformation indicates a reference for display of a product of thedetermined manufacturer. In this case, the reference display informationfor one product shelf rack 20 is prepared for each manufacturer.Therefore, the reference display information in FIG. 8 may not includethe manufacturer 506.

In a case where the reference display information is prepared for eachmanufacturer, the information processing apparatus 2000 may compare onlythe reference display information of the determined manufacturer withthe actual display information. In this case, for example, theinformation processing apparatus 2000 acquires information (for example,a manufacturer name) for determining the reference display informationwhich is a target of the comparison. This information is input by theuser of the information processing apparatus 2000. For example, the userperforms an input operation of specifying the manufacturer name withrespect to the above-described mobile terminal 3000. The application 100transmits the manufacturer name, which is input by the user, to beincluded in the above-described request. The information processingapparatus 2000 receives the request, and thus it is possible to graspthe manufacturer name specified by the user. In addition, for example,the user may input a manufacturer name by operating an input deviceconnected to the information processing apparatus 2000.

<Method for Acquiring Reference Display Information: S106>

There are various methods for acquiring the reference displayinformation by the information processing apparatus 2000. For example,the reference display information is stored in advance in the storageapparatus which is accessible from the information processing apparatus2000. In this case, the information processing apparatus 2000 acquiresthe reference display information by accessing the storage apparatus. Atthis time, the information processing apparatus 2000 acquires thereference display information relevant to the product shelf rack 20 byusing the identifier of the product shelf rack 20 which is a target ofprocessing. In a case where the reference display information of thedetermined manufacturer is acquired, the information processingapparatus 2000 acquires the reference display information of thedetermined manufacturer by using the manufacturer name or the like whichis acquired using the above-described method.

In addition, for example, the reference display information may bestored in the storage apparatus included in the management server. Inthis case, for example, the information processing apparatus 2000acquires the reference display information of the product shelf rack 20,which is determined by the identifier, by transmitting a requestincluding the identifier of the product shelf rack 20 to the managementserver. In a case where the reference display information of thedetermined manufacturer is acquired, the identifier of the manufacturer(for example, the manufacturer name) is also included in this request.

<Comparison of Actual Display Information with Reference DisplayInformation: S108>

The comparison unit 2040 compares the actual display information withthe reference display information, and generates comparison informationrepresenting a result of the comparison (S108). For this reason, thecomparison unit 2040 determines whether or not each reference (eachreference for the product, the group of products, or the manufacturer)indicated by the reference display information is satisfied in theactual display information.

For example, the comparison unit 2040 determines whether or not eachproduct, which is indicated in the reference display information, isindicated in the actual display information by comparing the productname (identification information of the product) indicated in the actualdisplay information with the product name (identification information ofthe product) indicated in the reference display information.

In addition, for example, in a case where the reference displayinformation indicates a position of a certain product, the comparisonunit 2040 determines whether or not the position of the product in theactual display information matches the position indicated in thereference display information.

In addition, for example, in a case where the reference displayinformation indicates the number of faces of a certain product, thecomparison unit 2040 determines whether or not the number of faces ofthe product indicated by the actual display information matches thenumber of faces indicated in the reference display information. Here,the number of faces of the product indicated by the actual displayinformation may be larger than the number of faces of the productindicated by the reference display information. For example, a situationmay arise in which, for a product that should have three faces, thenumber of faces is five in an actual display situation. In this way, ina case where the number of faces of the product in the actual displaysituation exceeds the number of faces indicated by the reference displayinformation, the product may be determined that the number of facesmatches the reference or does not match the reference. Information fordetermining a result of the determination on either matching ornon-matching is included in, for example, the reference displayinformation.

In a case where the reference display information indicates the parallelrequest or the arrangement request, the comparison unit 2040 determineswhether or not the parallel request or the arrangement request issatisfied on the basis of the position of each product indicated in theactual display information. For example, it is assumed that thereference display information indicates the parallel requestrepresenting that products, which belong to a certain product group,should be collectively placed. In this case, the comparison unit 2040determines whether or not the products of the product group arecollectively placed on the basis of the positions of the respectiveproducts, which belong to the product group, in the actual displayinformation. Specifically, in a case where the positions of therespective products, which belong to the product group, are continuousin the stage direction or in the column direction, the products arecollectively placed, and thus the parallel request is satisfied.Further, for example, it is assumed that the reference displayinformation indicates the arrangement request representing that aplurality of certain products should be placed adjacent to each other.In this case, the comparison unit 2040 determines whether or not thearrangement request is satisfied by determining whether or not thepositions of the plurality of products are adjacent in the actualdisplay information.

In the case where the reference display information indicates theapplicable period, the comparison unit 2040 compares only the productswhose applicable period includes a current time in the reference displayinformation, among the products indicated in the actual displayinformation, with the reference display information.

FIG. 9 is a diagram illustrating the comparison information. A table inFIG. 9 is referred to as a table 600. The table 600 includes columns ofa product 602, a group 604, a manufacturer 606, a display 608, aposition 610, the number of faces 612, a parallel request 614, and anarrangement request 616. A record, in which the product name isindicated in the product 602, indicates a result of the comparison ofthe reference of the product having the product name indicated in thereference display information with the actual display information. Arecord, in which the group name is indicated in the group 604, indicatesa result of the comparison of the reference of the product group of thegroup name indicated in the reference display information with theactual display information. A record, in which the manufacturer name isindicated in the manufacturer 606, indicates a result of the comparisonof the reference of the product of the manufacturer name indicated inthe reference display information with the actual display information.

Each of the display 608, the position 610, the number of faces 612, theparallel request 614, and the arrangement request 616 indicates whetheror not the actual display information matches the reference. Forexample, in the table 600, a case where the display 608 of a record of acertain product indicates “non-match” represents a case where the nameof the product is indicated in the reference display information, and,on the other hand, the name of the product is not indicated in theactual display information. That is, the case indicates that a productto be displayed on the product shelf rack 20 is not displayed. Inaddition, for example, in the table 600, a case where the position 610of the record of the certain product is “non-match” represents a casewhere the position of the product indicated in the actual displayinformation does not match the reference indicated in the referencedisplay information.

There are various methods for handling the comparison information. Forexample, the comparison unit 2040 stores the comparison information inthe storage apparatus. In addition, for example, the comparison unit2040 may generate image data or the like representing the result of thecomparison using the comparison information, as described in an exampleembodiment which will be described later.

<<Restriction of Comparison Items>>

It is assumed that the reference display information indicates referencefor a plurality of items such as the product name of the product to bedisplayed on the product shelf rack 20, the position of each product,and the number of faces of each product. In this case, the comparisonunit 2040 may compare the actual display information with the referencedisplay information for all items indicated in the reference displayinformation, or may compare the actual display information with thereference display information for some items. In the latter case, forexample, an item to be compared is specified by an input operationperformed by the user.

Further, the comparison unit 2040 may compare the actual displayinformation with the reference display information only for someproducts. For example, the comparison unit 2040 compares the actualdisplay information and the reference display information only for theproduct of the determined manufacturer. In this case, for example, thedetermined manufacturer is specified by an input operation performed bythe user.

A method for comparing the actual display information with the referencedisplay information only for the determined manufacturer is not limitedto the above-described method. For example, as described above, thereference display information may be prepared for each manufacturer, andonly the reference display information of the determined manufacturermay be acquired by the comparison unit 2040.

<<Computation of Match Rate>>

The comparison unit 2040 may compute an index value representing thedegree in which the display situation of the product represented by theactual display information matches the reference indicated in thereference display information. The index value is referred to as a matchrate. For example, it is possible to define the match rate as a valueacquired by dividing “the number of products, which are displayed on theproduct shelf rack 20 and which matches the reference indicated in thereference display information” by “a total number of products whichshould be displayed on the product shelf rack 20”. The followingEquation (1) represents the definition. Here, “a certain product matchesthe reference” means that the information of the product indicated bythe actual display information matches all the references (the position,the number of faces, and the like) indicated by the reference displayinformation.

$\begin{matrix}{{{match}{rate}} = \frac{{the}{number}{of}{products}{which}{match}{reference}}{{the}{number}{of}{products}{which}{should}{be}{displayed}{on}{shelf}}} & (1)\end{matrix}$

Further, the match rate may be computed not only for the individualproduct shelf rack 20 but also for an entire store. In this case, thematch rate of the entire store is computed using the comparisoninformation generated for each of the plurality of product shelves 20.Specifically, it is possible to define the match rate as a valueacquired by dividing “the number of products which are displayed in thestore and which match the reference indicated in the reference displayinformation” by “the total number of products to be displayed in thestore”. The following Equation (2) expresses the definition.

$\begin{matrix}{{{match}{rate}} = \frac{{the}{number}{of}{products}{which}{match}{reference}}{{the}{number}{of}{products}{which}{should}{be}{displayed}{in}{store}}} & (2)\end{matrix}$

Hereinafter, the match rate (Equation (1)) computed for each shelf rackis referred to as a “match rate for each shelf rack”, and the match rate(Equation (2)) computed for the entire store is referred to as a “matchrate for each store”. For example, in a case where one company manages aplurality of stores, it is possible to use the match rate for each storeas one of the indexes for grasping an operation situation of each store.It can be said that, in a store having a high match rate for each store,the products are appropriately displayed according to the reference. Onthe other hand, there is a possibility that, in a store having a lowmatch rate for each store, the products are not appropriately displayed.

Here, an opportunity, in which the information processing apparatus 2000computes the match rate for each store, is arbitrary. For example, theinformation processing apparatus 2000 uses a fact that the comparisoninformation is generated for all the product shelves 20 existing in thestore as a trigger and computes the match rate for each store for thestore.

It should be noted that, the match rate for each shelf rack and thematch rate for each store may be computed not for all the displayedproducts but only for the products of the determined manufacturer. Forexample, it is assumed that the match rate for each shelf rack iscomputed for a certain manufacturer. In this case, the comparison unit2040 computes a value acquired by dividing “the number of products thatmatch the reference indicated in the reference display information amongthe products, which are placed on the product shelf rack 20, of themanufacturer” by “the total number of products of the manufacturer to bedisplayed on the product shelf rack 20” as the match rate for each shelfrack for the manufacturer. In addition, the comparison unit 2040computes a value acquired by dividing the “number of products that matchthe reference indicated in the reference display information among theproducts of the manufacturer placed in the store” by “the number ofproducts of the manufacturer to be displayed in the store” as the matchrate for each store for the manufacturer.

In a case where, for the determined manufacturer, the match rate foreach shelf rack or the match rate for each store is computed, theinformation processing apparatus 2000 acquires identificationinformation (the manufacturer name, the manufacturer identifier, and thelike) for determining the manufacturer. The identification informationof the manufacturer is included in, for example, a request transmittedfrom the mobile terminal 3000.

<<Alert Using Match Rate>>

As described above, the information processing apparatus 2000 acquiresthe identifier of the product shelf rack 20 in order to identify theproduct shelf rack 20 included in the shelf rack image 12. Further, theinformation processing apparatus 2000 acquires the reference displayinformation for the product shelf rack 20 determined by the identifier.Here, in a case where the information processing apparatus 2000 acquiresan identifier of the product shelf rack 20, which is different from theproduct shelf rack 20 included in the shelf rack image 12, theinformation processing apparatus 2000 compares the display situation ofthe product shelf rack 20 included in the shelf rack image 12 with thereference display information for the different product shelf rack 20.Therefore, there is a high probability that the match rate for eachshelf rack, which is computed in this case, is low.

Therefore, the information processing apparatus 2000 may be configuredto determine whether or not the match rate for each shelf rack, which iscomputed by the comparison unit 2040, is equal to or less than apredetermined value, and to output an alert having a gist that there isa possibility that the identifier of the product shelf rack 20 isincorrect in a case where the match rate for each shelf rack is equal toor less than the predetermined value.

For example, it is assumed that the information processing apparatus2000 receives a request, which includes the shelf rack image 12 and theidentifier of the product shelf rack 20, from the mobile terminal 3000.In this case, in a case where the computed match rate for each shelfrack is equal to or larger than the predetermined value, the informationprocessing apparatus 2000 transmits a message representing theabove-described alert as a response with respect to the request. It ispossible for the user, who receives the message, of the mobile terminal3000 to take a measure of, for example, correcting transmission of therequest by specifying the identifier of the product shelf rack 20.

Although a case where the alert is output in a situation in which thereis a possibility that the identifier of the product shelf rack 20 isincorrect is exemplified in the above, the situation in which the alertis output is not limited to the situation. For example, in a case whereat least any one of the match rate for each shelf rack and the matchrate for each store, which are computed by the comparison unit 2040, isused and the match rate does not satisfy a predetermined condition, theinformation processing apparatus 2000 may output an alert representing agist that an appropriate display according to the reference is notperformed. For example, the predetermined condition is a condition thatthe match rate is equal to or larger than the predetermined value.

Second Example Embodiment

FIG. 10 is a block diagram illustrating a functional configuration of aninformation processing apparatus 2000 according to a second exampleembodiment. Except for the items described below, the informationprocessing apparatus 2000 according to the second example embodiment hasthe same functions as the information processing apparatus 2000according to the first example embodiment.

The information processing apparatus 2000 according to the secondexample embodiment includes a generation unit 2060. The generation unit2060 generates a result image for the product shelf rack 20 using thecomparison information generated for the product shelf rack 20. Theresult image of the product shelf rack 20 is image data representedproducts which do not match the reference indicated in the referencedisplay information and products that match the reference indicated inthe reference display information in a distinguishable aspect. Theresult image may be stored in the storage apparatus as an image file, ormay be displayed on the display apparatus as screen data.

According to the information processing apparatus 2000 of the exampleembodiment, a result, which is acquired by comparing a display situationof the products on the product shelf rack 20 with the reference displayinformation, is graphically represented in a form of the result image.Therefore, it is possible to intuitively and easily recognize adifference between the display situation of the products on the productshelf rack 20 and the reference for the display.

<Display Apparatus>

It is possible to use any display apparatus as the display apparatus onwhich the result image is displayed. For example, the display apparatusincludes a display apparatus connected to the information processingapparatus 2000 or a display apparatus of the mobile terminal 3000. Inaddition, for example, the display apparatus may be a terminal installedin a staff room of a store where the product shelf rack 20 is installed,or a display apparatus connected to a terminal installed in an office ofa company that manages the store. In addition, for example, it isassumed that a rounder or the like of a certain manufacturer uses theinformation processing apparatus 2000 in order to check whether or not aproduct of the manufacturer is correctly displayed on the product shelfrack 20. In this case, the display apparatus on which the result imageis displayed may be a display apparatus connected to the terminalinstalled in the office or the like of the manufacturer. The terminalinstalled in the staff room or the office is, for example, a PC.

Here, in a case where the display apparatus, on which the result imageis displayed, is not connected to the information processing apparatus2000, the information processing apparatus 2000 transmits the resultimage to the terminal to which the display apparatus is connected.Therefore, it is necessary for the information processing apparatus 2000to be able to determine a transmission destination of the result image.

There are various methods for determining the transmission destinationof the result image. For example, information for determining thetransmission destination of the result image is stored in the storageapparatus, which is accessible from the information processing apparatus2000, in advance. In this case, the information processing apparatus2000 determines the transmission destination of the result image byaccessing the storage apparatus.

In addition, for example, in a case where the information processingapparatus 2000 performs a process according to the request transmittedfrom the mobile terminal 3000, the information processing apparatus 2000transmits the result image to the transmission source of the request.The transmission destination of the result image may be included in therequest. In this case, the information processing apparatus 2000transmits the result image to the transmission destination indicated inthe request.

<Result Image>

Specific content of the result image is various. Hereinafter, variationsin the result image will be illustrated.

For example, the generation unit 2060 generates the result image bysuperimposing the superimposition data representing a result of thecomparison of the actual display information with the reference displayinformation on the shelf rack image 12. FIG. 11 is a first diagramillustrating the result image. In a result image 80 of FIG. 11 , thesuperimposition data representing the result of the comparison issuperimposed on the shelf rack image 12. Superimposition data 90 (arectangular in a dot pattern) is superimposed on an image region of theproduct which matches the reference indicated in the reference displayinformation. In addition, superimposition data 92 (a rectangle in anoblique pattern) is superimposed on an image region of a product whichdoes not match the reference indicated in the reference displayinformation. Further, superimposition data 94 (a rectangular in alattice pattern) is superimposed on the image region (that is, the imageregion representing a position of a shortage of the product)representing a position where the product should be displayed accordingto the reference display information, among the image regions where theproduct is not displayed. For example, a case where although thereference display information indicates a position of “the second stage,the third row” as the position of the product A, no product is disposedat the position may be referred to as the shortage of the product at theposition.

The superimposition data may use a color instead of a pattern such asthe dot pattern. For example, instead of the dot pattern, the obliquepattern, and the lattice pattern, respectively, a green color, a redcolor, and a yellow color are used.

Here, the superimposition data (the superimposition data 92 or thesuperimposition data 94 in FIG. 11 ) representing a part that does notmatch the reference indicated in the reference display information maybe emphasized using a method for blinking or the like.

The product that matches the reference and the product that does notmatch the reference may be distinguished by a shape of thesuperimposition data instead of the pattern or the color of thesuperimposition data. FIG. 12 is a second diagram illustrating theresult image 80. In the result image 80 of FIG. 12 , superimpositiondata 96 (a black circle) is superimposed on an image region of theproduct which does not match the reference indicated in the referencedisplay information. Further, superimposition data 97 (a cross) issuperimposed on an image region representing the position of theshortage of the product.

In addition, for example, a heat map may be used as the superimpositiondata. FIG. 13 is a third diagram illustrating the result image 80. InFIG. 13 , one superimposition data 98 is superimposed on the shelf rackimage 12. The superimposition data 98 is the heat map in which a colorof a part that does not match the reference display information isrelatively dark, and a color of a part that matches the referencedisplay information is relatively light.

The result image 80 may include a character string or the likerepresenting the match rate for each shelf rack, in addition to theshelf rack image 12 on which the superimposition data representing theresult of the comparison is superimposed. FIG. 14 is a diagramillustrating the result image 80 including the match rate for each shelfrack. The result image 80 of FIG. 14 includes a character string 70representing the match rate for each shelf rack on the shelf rack image12.

The result image 80 may include a list of products displayed on theshelf rack image 12 in addition to the shelf rack image 12 on which thesuperimposition data representing the result of the comparison issuperimposed. FIG. 15 is a diagram illustrating the result image 80including the list of the products. In FIG. 15 , the shelf rack image 12on which the superimposition data is superimposed is displayed in anarea 120. On the other hand, in the area 130, a product list 140 isdisplayed.

Here, in a case where the result image is displayed on the displayapparatus, the generation unit 2060 may change the display of the area120 according to an input operation performed by the user on the productlist 140. For example, it is assumed that the user selects a certainrecord in the product list 140. In this case, the comparison unit 2040is configured to emphasize an image region corresponding to a productindicated by the selected record in the area 120.

FIG. 16 is a diagram illustrating a state in which the result image 80changes according to an operation performed on the product list 140. InFIG. 16 , it is assumed that the user adds an input operation ofselecting a record of the product A on the result image 80 illustratedin FIG. 15 . According to the input operation, superimposition data 160is displayed on result image 80. The superimposition data 160 isacquired by extracting and enlarging an image region corresponding tothe product A from the result image 80 of FIG. 15 , and is displayed tobe located in front of the shelf rack image 12. Therefore, the imageregion corresponding to the product A selected by the user is emphasizedand viewed as being raised to the front. The operation of selecting therecord is not limited to a touch operation, and may be a mouse operationor the like.

In contrast, the generation unit 2060 may change the display of theproduct list 140 according to the input operation performed by the useron the product image displayed in the area 120. Specifically, in a casewhere the user selects an image of a certain product, the record of theproduct list 140 representing the product is emphasized.

FIG. 17 is a diagram illustrating a state in which the result image 80changes according to the operation performed on the product image. InFIG. 17 , a background color of a record representing the product A ischanged according to a fact that the user selects the image region 170representing the product A in the area 120.

<Method for Updating Display of Display Apparatus>

In a case where the display apparatus that displays the result image 80is connected to a terminal other than the information processingapparatus 2000, an input operation with respect to the result image 80is detected by the terminal. For example, in a case where the resultimage 80 is displayed on the display apparatus of the mobile terminal3000, the input operation with respect to the result image 80 isdetected by the mobile terminal 3000.

As described above, in a case where the input operation with respect tothe result image 80 is detected by the terminal other than theinformation processing apparatus 2000, it is possible to use variousmethods as a method for updating display of the display apparatusaccording to the input operation. For example, a terminal that hasdetected the input operation with respect to the result image 80transmits content of the input operation (for example, informationrelevant to the superimposition data that is a target of the inputoperation) to the information processing apparatus 2000. The informationprocessing apparatus 2000 generates the result image 80 whose content isupdated based on the received information, and transmits the resultimage 80 to the terminal. Then, the terminal displays the receivedresult image 80 on the display apparatus. Through the series of flows,the result image 80 displayed on the display apparatus is updated.

In addition, for example, a terminal, such as the mobile terminal 3000,may have a function of updating the display of the display apparatusaccording to the input operation with respect to the result image 80. Inthis case, in a case where the terminal detects the input operationperformed by the user, the terminal changes the result image 80 byitself without transmitting content of the input operation to theinformation processing apparatus 2000. In this case, an application forbrowsing the result image 80 is installed in the terminal in advance.For example, in a case of the mobile terminal 3000, the above-describedapplication 100 has a function of browsing the result image 80.

<Case where Product that Cannot be Recognized from Shelf Rack Image 12Exists>

There may be a case where it is not possible to recognize a part of theproduct included in the shelf rack image 12 because a part of the shelfrack image 12 is unclear. In this case, the actual display informationmay indicate a position where some undetermined products (unknownproducts) are placed.

In a case where the actual display information indicates a position ofthe unknown product, it is preferable that the result image 80 indicatesthe position of the unknown product. Specifically, the generation unit2060 includes the superimposition data representing the unknown productin the result image 80. FIG. 18 is a diagram illustrating the resultimage 80 including the superimposition data representing the unknownproduct. In FIG. 18 , the superimposition data 110 represents theunknown product. That is, the result image 80 of FIG. 18 represents thatthe product at the position where the superimposition data 110 issuperimposed is not determined.

For the unknown product, it is preferable to perform imaging with thecamera 10 again. Therefore, for example, the superimposition data 110may be realized as a button for calling a camera function. In this case,for example, the result image 80 is displayed on the display apparatusof the mobile terminal 3000 provided with the camera 10. In this case,the application 100 detects a fact that the user presses thesuperimposition data 110, or the like, and in response thereto activatesa camera application that controls the camera 10. In a case where theuser operates the camera application to cause the camera 10 to performimaging, the application 100 transmits the generated shelf rack image 12to the information processing apparatus 2000. The information processingapparatus 2000 generates the actual display information again using theacquired shelf rack image 12.

Here, in a case where a product that was the unknown product is includedto be recognizable (clear) in the newly generated shelf rack image 12,the product name or the like of the product that was the unknown productis indicated in the newly generated actual display information.Therefore, the information processing apparatus 2000 generates thecomparison information again by comparing the newly generated actualdisplay information with the reference display information, and updatesthe result image 80. At this time, the information processing apparatus2000 may generate the comparison information only for the product thatis the unknown product in the previously generated actual displayinformation, and may merge the comparison information with thepreviously generated comparison information.

It should be noted that, a method for using the superimposition data 110representing the unknown product is not limited to a method forperforming imaging by the camera 10 again. For example, a configurationmay be used in which the superimposition data 110 includes informationrelevant to a product to be placed at the position of the unknownproduct. A display aspect of the information relevant to the product mayuse the same configuration as the information (correction information150-1 and correction information 150-2) illustrated in FIG. 19 whichwill be described later. In this case, it is possible to realize theinformation processing apparatus 2000 by obtaining information relevantto the product associated with the position of the unknown product fromthe reference display information. In a case where information relevantto the displayed product is selected by the user, the informationprocessing apparatus 2000 may generate actual display information, whichis different from the actual display information, again (that is,information relevant to the selected product is included in the actualdisplay information) according to the selection based on the informationrelevant to the selected product and the actual display informationgenerated using the shelf rack image 12 already acquired.

<Display of Information for Correcting Display of Product>

The result image 80 may include information representing the referenceindicated in the reference display information (that is, informationrepresenting how to correct the display situation so as to match thereference) for the part that does not match the reference indicated inthe reference display information. Hereinafter, the information isreferred to as the correction information.

FIG. 19 is a diagram illustrating the correction information. The resultimage 80 of FIG. 19 includes two pieces of correction information 150.When the correction information 150-1 is viewed, it is understood that aproduct placed at a position where the superimposition data 92 issuperimposed should be replaced with the product A. Also, when thecorrection information 150-2 is viewed, it is understood that theproduct B should be placed in a place where the superimposition data 94is superimposed.

Here, in a case where the result image 80 is displayed on the displayapparatus, the generation unit 2060 may include the correctioninformation 150 in the result image 80 from the beginning, or mayinclude the correction information 150 in the result image 80 at thetiming when the input operation is added with respect to thesuperimposition data (the superimposition data 92 and thesuperimposition data 94) which indicates not matching the reference.

In addition, the generation unit 2060 may generate the screen datarepresenting detailed information relevant to the product indicated inthe correction information 150 according to a fact that the inputoperation is added to the correction information 150. The informationrelevant to the product includes, for example, an identifier of theproduct, a group to which the product belongs, a manufacturer of theproduct, and a place where the product is stored (a place of a shelfrack installed in a warehouse and a position of the product on the shelfrack). For example, in a case of the shortage of the product, it isnecessary to bring the product from a storage location and replenish theproduct. Thus, the storage location of the product is displayed, andthus it is possible to easily perform an operation of replenishing theproduct.

Here, the correction information 150 may be displayed for all productsthat do not match the reference indicated in the reference displayinformation, or may be displayed for some products. In the latter case,whether or not to display the correction information 150 for a productis determined based on a degree of importance of the product. In thiscase, the degree of importance is set for each product in advance.

For example, the generation unit 2060 generates the correctioninformation 150 only for products whose degree of importance is equal toor larger than a predetermined value, and includes the correctioninformation 150 in the result image 80. In addition, for example, thegeneration unit 2060 may generate the correction information 150 onlyfor a predetermined number of products at a high degree of importanceamong the products that do not match the reference indicated in thereference display information, and include the correction information150 in the result image 80.

Example of Hardware Configuration

Similar to the first example embodiment, a hardware configuration of thecomputer that realizes the information processing apparatus 2000according to the second example embodiment is illustrated in, forexample, FIG. 3 . However, the storage device 1080 of the computer 1000that realizes the information processing apparatus 2000 of the exampleembodiment further stores a program module that realizes the functionsof the information processing apparatus 2000 of the example embodiment.

Hereinabove, although the example embodiments of the present inventionare described with reference to the accompanying drawings, the exampleembodiments are examples of the present invention, and it is possible touse a combination of the respective example embodiment or to use variousconfigurations other than the above-described configurations.

Although a part or all of the above-described example embodiments may bedescribed as the supplementary below, the present invention is notlimited thereto.

1. An information processing apparatus including:

an image analysis unit that performs image analysis on a shelf rackimage in which a product shelf rack is captured, and generates actualdisplay information representing a display situation of a product on theproduct shelf rack; and

a comparison unit that compares the actual display information withreference display information representing a reference for display ofthe product on the product shelf rack, and generates comparisoninformation representing a result of the comparison.

2. The information processing apparatus according to 1,

in which the reference display information indicates identificationinformation of each product to be displayed on the product shelf rack,and

in which the image analysis unit generates the actual displayinformation representing identification information of each productdisplayed on the product shelf rack.

3. The information processing apparatus according to 2,

in which the reference display information indicates at least one of adisplay position of the product on the product shelf rack, a number offaces, and a disposition relationship with another product.

4. The information processing apparatus according to any one of 1 to 3,in which the comparison unit computes a match rate representing a degreeat which the display situation of the product matches the referenceindicated in the reference display information, and causes the matchrate to be included in the comparison information.

5. The information processing apparatus according to any one of 1 to 3,in which the comparison unit acquires an identifier of a manufacturer,computes a match rate representing a degree at which a display situationof a product of the manufacturer determined by the identifier matchesthe reference indicated in the reference display information, and causesthe match rate to be included in the comparison information.

6. The information processing apparatus according to any one of 1 to 5further including a generation unit that generates a result imagerepresenting a product, which does not match the reference indicated inthe reference display information, using the comparison information.

7. The information processing apparatus according to 6, in which theresult image represents the product that does not match the referenceindicated in the reference display information and a product thatmatches the reference indicated in the reference display information ina distinguishable aspect.

8. The information processing apparatus according to 6 or 7, in whichthe generation unit, for the product that does not match the referenceindicated in the reference display information, causes correctioninformation representing a position, at which the product is to bedisplayed, or a product to be displayed at the position at which theproduct is displayed, to be included in the result image.

9. An information processing system including an information processingapparatus and a mobile terminal,

in which the mobile terminal transmits a shelf rack image, in which aproduct shelf rack is captured, to the information processing apparatus,and

in which the information processing apparatus includes

an image analysis unit that receives the transmitted shelf rack image,performs image analysis on the shelf rack image, and generates actualdisplay information representing a display situation of the product onthe product shelf rack,

a comparison unit that compares the actual display information withreference display information representing a reference for display ofthe product on the product shelf rack, and generates comparisoninformation representing a result of the comparison, and

a generation unit that generates a result image representing a productwhich does not match the reference indicated in the reference displayinformation, and transmits the generated result image to the mobileterminal, and

in which the mobile terminal receives the result image and displays theresult image on a display apparatus.

10. The information processing system according to 9, in which thereference display information generates the actual display informationwhich indicates at least one of a display position of the product on thesaid product shelf rack, a number of faces, and a dispositionrelationship with another product.

11. The information processing system according to 10, in whichreference display information indicates the display position of theproduct on the product shelf rack, a number of faces, or the dispositionrelationship with another product.

12. The information processing system according to any one of 9 to 11,in which the comparison unit computes a match rate representing a degreeat which the display situation of the product matches the referenceindicated in the reference display information, and causes the matchrate to be included in the comparison information.

13. The information processing system according to any one of 9 to 11,in which the comparison unit acquires an identifier of a manufacturer,computes a match rate representing a degree at which a display situationof a product of the manufacturer determined by the identifier matchesthe reference indicated in the reference display information, and causesthe match rate to be included in the comparison information.

14. The information processing system according to any one of 9 to 13,in which the result image represents the product that does not match thereference indicated in the reference display information and a productthat matches the reference indicated in the reference displayinformation in a distinguishable aspect.

15. The information processing system according to any one of 9 to 14,in which the generation unit, for the product that does not match thereference indicated in the reference display information, causescorrection information representing a position, at which the product isto be displayed, or a product to be displayed at the position at whichthe product is displayed, to be included in the result image.

16. A control method executed by a computer, the method including:

an image analysis step of performing image analysis on a shelf rackimage in which a product shelf rack is captured, and generating actualdisplay information representing a display situation of a product on theproduct shelf rack; and

a comparison step of comparing the actual display information withreference display information representing a reference for display ofthe product on the product shelf rack, and generating comparisoninformation representing a result of the comparison.

17. The control method according to 16, in which the reference displayinformation indicates identification information of each product to bedisplayed on the product shelf rack, and

in which the image analysis step includes generating the actual displayinformation representing identification information of each productdisplayed on the product shelf rack.

18. The control method according to 17, in which the reference displayinformation indicates at least one of a display position of the producton the product shelf rack, a number of faces, and a dispositionrelationship with another product.

19. The control method according to any one of 16 to 18, in which thecomparison step includes comparing includes computing a match raterepresenting a degree at which the display situation of the productmatches the reference indicated in the reference display information,and causing the match rate to be included in the comparison information.

20. The control method according to any one of 16 to 18, in which thecomparison step includes acquiring an identifier of a manufacturer,computing a match rate representing a degree at which a displaysituation of a product of the manufacturer determined by the identifiermatches the reference indicated in the reference display information,and causing the match rate to be included in the comparison information.

21. The control method according to any one of 16 to 20, furtherincluding a generation step of generating a result image representing aproduct, which does not match the reference indicated in the referencedisplay information, using the comparison information.

22. The control method according to 21, in which the result imagerepresents the product that does not match the reference indicated inthe reference display information and a product that matches thereference indicated in the reference display information in adistinguishable aspect.

23. The control method according to 21 or 22, in which the generationstep, for the product that does not match the reference indicated in thereference display information, includes causing correction informationrepresenting a position, at which the product is to be displayed, or aproduct to be displayed at the position at which the product isdisplayed, to be included in the result image.

24. A program for causing a computer to execute respective steps of thecontrol method according to any one of 16 to 23.

1. An information processing apparatus comprising: at least one memory configured to store instructions; and at least one processor configured to execute the instructions to: perform image analysis on a shelf rack image in which a product shelf rack is captured by a camera, and generate actual display information representing a display situation of a product on the product shelf rack; perform a comparison of the actual display information with reference display information representing a reference for display of each product on the product shelf rack, and generate comparison information representing a result of the comparison; and generate a result image representing a non-compliant product using the comparison information, the non-compliant product not matching the reference for display of each product on the product shelf rack indicated in the reference display information.
 2. The information processing apparatus according to claim 1, wherein the result image represents the non-compliant product and a compliant product in a distinguishable aspect, the compliant product matching the reference for display of each product on the product shelf rack indicated in the reference display information.
 3. The information processing apparatus according to claim 2, wherein the at least one processor is further configured to execute the instructions to generate the result image by superimposing first superimposition data on an image region of the compliant product, and by superimposing second superimposition data on an image region of the non-compliant product, and the first superimposition data differs from the second superimposition data in at least one of patterns and colors.
 4. The information processing apparatus according to claim 2, wherein the at least one processor is further configured to execute the instructions to generate the result image by superimposing first superimposition data on an image region of the compliant product, and by superimposing second superimposition data on an image region of the non-compliant product, and the first superimposition data differs from the second superimposition data in shapes.
 5. The information processing apparatus according to claim 1, wherein the at least one processor is further configured to execute the instructions to compute a match rate representing a degree at which the display situation of the product on the product shelf rack matches the reference for display of each product on the product shelf rack indicated in the reference display information, and cause the match rate to be included in the comparison information.
 6. The information processing apparatus according to claim 5, wherein the at least one processor is further configured to execute the instructions to generate the result image by superimposing, on the shelf rack image, superimposition data using a heat map in which a color of a part that does not match the reference display information is relatively dark, and a color of a part that matches the reference display information is relatively light.
 7. The information processing apparatus according to claim 5, wherein the result image includes a character string representing the match rate for the shelf rack and the shelf rack image on which superimposition data representing the result of the comparison is superimposed.
 8. The information processing apparatus according to claim 1, wherein the at least one processor is further configured to execute the instructions to acquire an identifier of a manufacturer, compute a second match rate representing a degree at which a display situation of a product of the manufacturer determined by the identifier matches the reference for display of each product on the product shelf rack indicated in the reference display information, and cause the second match rate to be included in the comparison information.
 9. The information processing apparatus according to claim 1, wherein the at least one processor is further configured to execute the instructions to, for the non-compliant product, cause correction information representing a corrected position, at which the non-compliant product is to be displayed, or a replacement product to be displayed at a current position at which the non-compliant product is displayed, to be included in the result image.
 10. An information processing system comprising: an information processing apparatus; and a mobile terminal, wherein the mobile terminal transmits a shelf rack image, in which a product shelf rack is captured by a camera, to the information processing apparatus, wherein the information processing apparatus comprises at least one memory configured to store instructions; and at least one processor configured to execute the instructions to: receive the transmitted shelf rack image, perform image analysis on the shelf rack image, and generate actual display information representing a display situation of a product on the product shelf rack, perform a comparison of the actual display information with reference display information representing a reference for display of each product on the product shelf rack, and generate comparison information representing a result of the comparison, and generate a result image representing a non-compliant product using the comparison information, the non-compliant product not matching the reference for display of each product on the product shelf rack indicated in the reference display information, and transmit the generated result image to the mobile terminal, and wherein the mobile terminal receives the result image and displays the result image on a display apparatus.
 11. A control method executed by a computer, the control method comprising: performing image analysis on a shelf rack image in which a product shelf rack is captured by a camera, and generating actual display information representing a display situation of a product on the product shelf rack; comparing the actual display information with reference display information representing a reference for display of each product on the product shelf rack, and generating comparison information representing a result of the comparison; and generating a result image representing a non-compliant product using the comparison information, the non-compliant product not matching the reference for display of each product on the product shelf rack indicated in the reference display information. 